Goto

Collaborating Authors

 Surgery


ClinicalLab: Aligning Agents for Multi-Departmental Clinical Diagnostics in the Real World

Neural Information Processing Systems

Large language models (LLMs) have achieved significant performance progress in various natural language processing applications. However, LLMs still struggle to meet the strict requirements for accuracy and reliability in the medical field and face many challenges in clinical applications. Existing clinical diagnostic evaluation benchmarks for evaluating medical agents powered by LLMs have severe limitations. Firstly, most existing medical evaluation benchmarks face the risk of data leakage or contamination.


EgoExOR: An Ego-Exo-Centric Operating Room Dataset for Surgical Activity Understanding

Neural Information Processing Systems

Operating rooms (ORs) demand precise coordination among surgeons, nurses, and equipment in a fast-paced, occlusion-heavy environment, necessitating advanced perception models to enhance safety and efficiency. Existing datasets either provide partial egocentric views or sparse exocentric multi-view context, but do not explore the comprehensive combination of both. We introduce EgoExOR, the first OR dataset and accompanying benchmark to fuse first-person and thirdperson perspectives. Spanning 94 minutes (84,553 frames at 15 FPS) of two emulated spine procedures, Ultrasound-Guided Needle Insertion and Minimally Invasive Spine Surgery, EgoExOR integrates egocentric data (RGB, gaze, hand tracking, audio) from wearable glasses, exocentric RGB and depth from RGB-D cameras, and ultrasound imagery. Its detailed scene graph annotations, covering 36 entities and 22 relations (568,235 triplets), enable robust modeling of clinical interactions, supporting tasks like action recognition and human-centric perception. We evaluate the surgical scene graph generation performance of two adapted state-of-the-art models and offer a new baseline that explicitly leverages EgoExOR's multimodal and multi-perspective signals. This new dataset and benchmark set a new foundation for OR perception, offering a rich, multimodal resource for next-generation clinical perception.


SonoGym: High Performance Simulation for Challenging Surgical Tasks with Robotic Ultrasound

Neural Information Processing Systems

Ultrasound (US) is a widely used medical imaging modality due to its real-time capabilities, non-invasive nature, and cost-effectiveness. Robotic ultrasound can further enhance its utility by reducing operator dependence and improving access to complex anatomical regions. For this, while deep reinforcement learning (DRL) and imitation learning (IL) have shown potential for autonomous navigation, their use in complex surgical tasks such as anatomy reconstruction and surgical guidance remains limited -- largely due to the lack of realistic and efficient simulation environments tailored to these tasks. We introduce SonoGym, a scalable simulation platform for complex robotic ultrasound tasks that enables parallel simulation across tens to hundreds of environments. Our framework supports realistic and real-time simulation of US data from CT-derived 3D models of the anatomy through both a physics-based and a generative modeling approach.


EgoExOR: An Ego-Exo-Centric Operating Room Dataset for Surgical Activity Understanding

Neural Information Processing Systems

Operating rooms (ORs) demand precise coordination among surgeons, nurses, and equipment in a fast-paced, occlusion-heavy environment, necessitating advanced perception models to enhance safety and efficiency. Existing datasets either provide partial egocentric views or sparse exocentric multi-view context, but do not explore the comprehensive combination of both. We introduce EgoExOR, the first OR dataset and accompanying benchmark to fuse first-person and third-person perspectives. Spanning 94 minutes (84,553 frames at 15 FPS) of two emulated spine procedures, Ultrasound-Guided Needle Insertion and Minimally Invasive Spine Surgery, EgoExOR integrates egocentric data (RGB, gaze, hand tracking, audio) from wearable glasses, exocentric RGB and depth from RGB-D cameras, and ultrasound imagery. Its detailed scene graph annotations, covering 36 entities and 22 relations (568,235 triplets), enable robust modeling of clinical interactions, supporting tasks like action recognition and human-centric perception. We evaluate the surgical scene graph generation performance of two adapted state-of-the-art models and offer a new baseline that explicitly leverages EgoExOR's multimodal and multi-perspective signals. This new dataset and benchmark set a new foundation for OR perception, offering a rich, multimodal resource for next-generation clinical perception.


'You can't control everything': the rise in plastic surgeons asked to create 'AI face'

The Guardian

'You can't control everything': the rise in plastic surgeons asked to create'AI face' Growing numbers of people are seeking improbable cosmetic surgery based on chatbots' recommendations Plastic surgeons are increasingly concerned about the rise of "AI face", as more and more clients arrive in their offices with unrealistic AI-generated visions of what they want to look like. Dr Nora Nugent, a cosmetic surgeon from Tunbridge Wells, has seen this first hand. Clients have started coming to her office with photos of themselves beautified by AI and a false expectation that those results are achievable with surgery. She is also the president of the British Association of Aesthetic Plastic Surgeons, and says many colleagues are having similar experiences. "I can only predict an increase, given the rate AI has been incorporated into every aspect of life," she said.


Three near-death experiences that convinced doctors the soul may exist

Daily Mail - Science & tech

SNL season finale cold open sees ghost of Jeffrey Epstein played by Will Ferrell'haunt' Trump as dark jokes leave viewers shocked Jordon Hudson blasts double standards over Mike Vrabel and Dianna Russini'affair' scandal: 'What is going on?' No one wants to hang out with her': Why Meghan and Harry have been ditched by A-list friends as insiders reveal Oprah's merciless snub, why the Clooneys now want nothing to do with them - and how SHE'S the problem Truth about Kate Middleton's past before Prince William... we Americans see this for what it is: KENNEDY Kim Kardashian roasted over'ridiculous' outfit at Gucci show as she sits front row with Anna Wintour and Mariah Carey I was on track to make $1 million... then I quit my job and moved into an off-grid tiny home with no running water or electricity Professional tasters decide best and worst fast food cheeseburger - do you agree? Hamptons cancer cluster: Rates are spiking in summer enclave of New York's wealthy elite... and doctors think they know the tragic reason why Disturbing trove of images woke Los Angeles mayor Karen Bass doesn't want you to see: Filthy truth is so much worse than people think... Taylor Swift dazzles in glittering gown as she and Travis Kelce steal the spotlight at friend's wedding in NYC Golf star becomes instant fan favorite after stopping to smoke a cigarette with crowd in the middle of the PGA Championship: 'Man of the people' New kind of penis enlargement surgery will add inches, claims the doctor set to offer it... but there is a gruesome detail that may make some think twice She was every bit the adoring mother... then a leaked video exposed a'sadistic' secret even cops said'will bring tears to your eyes' I saw a 40-year-old middle-class mom in a psychiatric ward after a single hit of this drug. Her symptoms were terrifying but it's so common now... here's what you must know: DR MAX PEMBERTON Expert reveals the best way to cut the bread - and why you should never leave a'hinge' 'I saw things I can never unsee': Man who snuck into Air India crash morgue reveals what he saw... why it could blow apart the pilot suicide theory... and what happened when we visited the lone survivor Many people have reported near-death experiences, but in some cases, survivors appeared to bring back something far more unsettling than memories. Some survivors claimed they saw and heard things that should have been impossible while they were clinically dead, including conversations in operating rooms and objects located far outside their hospital beds. Several of the most famous cases involved patients whose brains allegedly showed little or no measurable activity at the time of their experiences.


Uncertainty Estimation for Safety-critical Scene Segmentation via Fine-grained Reward Maximization

Neural Information Processing Systems

Uncertainty estimation plays an important role for future reliable deployment of deep segmentation models in safety-critical scenarios such as medical applications. However, existing methods for uncertainty estimation have been limited by the lack of explicit guidance for calibrating the prediction risk and model confidence. In this work, we propose a novel fine-grained reward maximization (FGRM) framework, to address uncertainty estimation by directly utilizing an uncertainty metric related reward function with a reinforcement learning based model tuning algorithm. This would benefit the model uncertainty estimation through direct optimization guidance for model calibration. Specifically, our method designs a new uncertainty estimation reward function using the calibration metric, which is maximized to fine-tune an evidential learning pre-trained segmentation model for calibrating prediction risk.


Off-Policy Evaluation for Human Feedback

Neural Information Processing Systems

Off-policy evaluation (OPE) is important for closing the gap between offline training and evaluation of reinforcement learning (RL), by estimating performance and/or rank of target (evaluation) policies using offline trajectories only. It can improve the safety and efficiency of data collection and policy testing procedures in situations where online deployments are expensive, such as healthcare. However, existing OPE methods fall short in estimating human feedback (HF) signals, as HF may be conditioned over multiple underlying factors and is only sparsely available; as opposed to the agent-defined environmental rewards (used in policy optimization), which are usually determined over parametric functions or distributions. Consequently, the nature of HF signals makes extrapolating accurate OPE estimations to be challenging. To resolve this, we introduce an OPE for HF (OPEHF) framework that revives existing OPE methods in order to accurately evaluate the HF signals. Specifically, we develop an immediate human reward (IHR) reconstruction approach, regularized by environmental knowledge distilled in a latent space that captures the underlying dynamics of state transitions as well as issuing HF signals. Our approach has been tested over two real-world experiments, adaptive in-vivo neurostimulation and intelligent tutoring, as well as in a simulation environment (visual Q&A). Results show that our approach significantly improves the performance toward estimating HF signals accurately, compared to directly applying (variants of) existing OPE methods.


SurgicAI: A Hierarchical Platform for Fine-Grained Surgical Policy Learning and Benchmarking

Neural Information Processing Systems

Despite advancements in robotic-assisted surgery, automating complex tasks like suturing remains challenging due to the need for adaptability and precision. Learning-based approaches, particularly reinforcement learning (RL) and imitation learning (IL), require realistic simulation environments for efficient data collection. However, current platforms often include only relatively simple, non-dexterous manipulations and lack the flexibility required for effective learning and generalization. We introduce SurgicAI, a novel platform for development and benchmarking that addresses these challenges by providing the flexibility to accommodate both modular subtasks and more importantly task decomposition in RL-based surgical robotics. Compatible with the da Vinci Surgical System, SurgicAI offers a standardized pipeline for collecting and utilizing expert demonstrations. It supports the deployment of multiple RL and IL approaches, and the training of both singular and compositional subtasks in suturing scenarios, featuring high dexterity and modularization. Meanwhile, SurgicAI sets clear metrics and benchmarks for the assessment of learned policies. We implemented and evaluated multiple RL and IL algorithms on SurgicAI. Our detailed benchmark analysis underscores SurgicAI's potential to advance policy learning in surgical robotics.


Remote robot surgery removes cancer 1,500 miles away

FOX News

A London doctor controlled robotic surgical arms in Gibraltar to perform prostate cancer telesurgery in near real time from 1,500 miles away.